Structural Parsing of Natural Language Text in Tamil Language Using Dependency Language Model

Parsing is an important process of Natural Language Processing (NLP) and Computational Linguistics which is used to understand the syntax and semantics of a natural language sentences confined to the grammar. Parser is a computational system which processes input sentence according to the productions of the grammar, and builds one or more constituent structures which conform to the grammar. The interpretation of natural language text depends on the context also. Statistical methods mainly focus on semantics where as structural methods focus on syntax. Though statistical parsing gives the better performance through N-gram probabilities and large vocabulary size, it has some disadvantages like lack of support in free ordering of words and long term relationship. Structural parsing provides solutions to some extent but it is very tedious for larger vocabulary corpus. To accommodate syntax and semantics, structural component is to be involved in statistical approach. To add the structural component and balance the vocabulary size, Lexicalized and Statistical Parsing (LSP) is to be employed with phrase or dependency structure language model. To maintain the long term relationship in complex and large sentences, phrase structure language model is not suitable. Also large training set is needed to leverage the performance. When dependency relations are applied among words, direct relationships can be established. This paper focuses on dependency based lexicalized and statistical parsing of natural language text in Tamil language. LSP needs dependency based hybrid language model which supports better degree of long term relationship and free ordering of words than phrase structure. New Part of Speech (POS) tag set for Tamil language has been proposed with more than 500 tags which has the wider coverage. Also dependency tag set with 31 tags is used for obtaining dependency relations. Medium size vocabulary dependency Treebank has been developed with 326 sentences which comprises more than 5000 words with manual annotation. Dependency language model has been trained using dependency Treebank. This structural parser gives better results and covers above said features.